 |
AFO 133 - Input /output profiles
Use this AFO to create and maintain profiles for importing records into Cataloguing databases, and exporting records from Cataloguing databases.
The import/export module can handle multiple digital file formats. Below are descriptions of several varieties of raw data.
1. “Delimited”
A delimited file contains data separated by specific characters. Usually more than one separator is used, at least one for separating data fields and one for separating records.
In the example we see an Excel sheet with bibliographic data. From Excel (or Access) it is relatively simple to save the delimited data.
Example:
A file with delimited data (with “;” as delimiter):
Title; author; place of publication; publisher; etc.
The import module processes the data by column as they were saved in Excel or Access, where column 4 in the example is the title, column 3 the author etc.
The only restriction the import module has is that each column is a filed. You cannot spread data from one column over multiple fields. In practice you often see all authors of a title in one column. For a successful import the primary author and all subsequent authors should be put in separate columns.
2. “Tagged”
A tagged format implies each data field is preceded by a label that allows the import module to determine what type of data is in it.
An example of a pure tagged format is:
The first two letters on each line determine the type of field. So: LA = language, MT = material type, TI = title, DE = descriptor, IT = item details, etc. After the space you have the contents of the filed. The dollar sign means ‘end of record’. This must be an ASCII text file which will look like the above example when opened with Notepad.
A tagged file can also contain tekst of variable lenght. Like:
In this case you need a special character to mark the end of the tag and the beginning of the data. IN the example this is a pipe, “|”. A space would not be sufficient, because the data (field content) can also contain spaces.
For both delimited and tagged formats you must determine what the separator characters are going to be for fields and records. The import module needs to know the ASCII value of those characters.
3. “XML”
Exporting and importing records in XML format is incorporated in the standard facilities for data export (AFO 132) and import (AFO 131). It applies to both bibliographic documents and authority records. Both export and import use the profiles created and modified through AFO 133, which in its turn optionally use conversion profiles defined through AFO 134. All this implies that XML records are imported in exactly the same manner as ISO2709, delimited or tagged records.
If you have a Vubis Smart database using one of the three most common formats used within Vubis Smart (MARC21, UNIMARC and Smart), you have the following possibilities:
· Export and import of MARCXML records.
- Use an import/export profile that uses the file format MARCXML.
· Export and import of Dublin Core in XML format.
- Use an import/export profile that uses the file format XML Dublin Core and use as a conversion profile one of the predefined XML Dublin Core conversion profiles.
· Export and import of other XML formats.
- There are multiple possibilities here:
· Create your own conversion profile in AFO 134 and create an import/export profile in AFO 133 that uses "XML Generic" as file format. Under "record delimiter", fill in the name of the element that indicates the start of a new record.
· Import/export ISO 2709 records and use a conversion tool outside of Vubis Smart.
· Use techniques outside of Vubis Smart to convert MARCXML or DC-XML to or from the desired format. Standard techniques such as XSLT can be used to create such conversions.
An example of Dublin Core XML:
After you have chosen this AFO an overview screen with existing profiles will be presented. You can edit existing profiles on this screen or create new ones.
: click this icon to create a new profile. You will first be presented with an imput form where you must enter the profile name and select an application (bibliographic or authority).
: select an existing profile and click on this icon to modify the profile.
: select an existing profile and click on this icon to delete the profile.
: select an existing profile and click on this icon to create a new profile by copying the existing one.
After selecting an existing profile or creating a new one, you are presented with an input form with 5 tabs. All tabs must be filled out. They are described in more detail below.
The first tab contains the general characteristics of the input/output file.
Character set of external data: Select a standard character set from the dropdown list. If you are not sure, just choose one and look at the result in a test load. If the result is incorrect (for example strange characters appear) you can adjust by choosing another character set.
Language for language-dependant fields: Select a language form the dropdown list. Usually this is set to <all languages>.
Code of field with record ID: In the example this is 1. This means the loader will assume the first field will contain a record number. In MARC21 tag 001 contains the control number. This can then be used to store separately. Because Vubis Smart has its own record ID numbers, the external record ID is not used as such. But it can be useful to retain the information in the record
Code of subfield with record ID: Here you can specify the specific subfield that contains a record ID (optional).
Record format conversion: This is one of the most critical steps. Select a profile form the dropdown list. The profiles are defined in AFO 134. You can do that later and then come back to this tab to add it.
Character for start of non-filing part: To denote where the non-filing character must be positioned in Vubis Smart. Within the Smart format you can use this to specify a certain part of the title must not be indexed. Enter the ASCII value of the non-filing character.
Character for end of non-filing part: See above.
Maximum program errors for conversion: When the process reaches the number specified here, the loading will be stopped. The system checks such details as valid fields for the format as specified in AFO 151. When for instance ‘language’ is a mandatory field for the format, but the incoming records contain no language then this will be seen as an error.
The second tab allows for specification of the various delimiters.
File format: Select a file format from the dropdown list.
Record delimiter: Enter the ASCII value of the separator character. Say you import an Excel file. When exporting the Excel file to a text file add the delimiters (like comma or semicolon).
Field delimiter: See above.
Subfield delimiter: See above. Subfields are not recognised in delimited files.
Tag delimiter: See above. This can be the space (ASCII value 32) between tag number and data.
Tag length: For MARC21 this should be 3 for instance.
Fixed field map: This is only used when all data can be found on fixed positions in a file.
Quotes around data: denotes whether or not the loader must take quotes into account.
The third tab offers the opportunity to add matching and merging profiles.
Matching profile: Select a profile from the dropdown list. These are used to specify what must happen when records are loaded that already exist in the database. You can for instance specify that the incoming record prevails. You need to create the matching and merging profiles in AFO 114 and 115.
Matching profile: See above.
Database: When you have multiple databases, you need to specify in which database the records must be loaded. Select one from the dropdown list.
Default template: Select one from the dropdown list (the templates must be defined in AFO 153, 154).
Maximum errors: these three fields determine fault tolerance.
Record status: Choose the correct statuses from the dropdown list.
If data match rejected form: When records are loaded through AFO 131 with the record loader it may occur that the contents of an authority controlled bibliographic field match with an existing rejected form in the related authority database. Here you can specify what must happen in such a case. Three different options can be chosen if the data match a rejected form:
· Create new main heading: The rejected form is removed from the database, and instead a new main heading will be created
· Discard data: The data are removed from the import record – this is the current situation
· Link to main heading: The field is linked to the main heading of the rejected form
In all three cases a warning message will be added to the loader report.
This tab allows various settings for creating savelists of not loaded records, loaded records, etc.
Specify whether the savelist must be created for a specific user (* means current user). You can select existing savelist names or type in new names. And you can choose to append the date to the savelist name.
On this tab you can create savelists for imports that include items. By entering a period and setting the savelist to read-only you can eventually delete these items from the system. These options are only valid for import. The last option is only valid for export, here you determine from which institution(s) you export item data.
Note
Items will be exported with a status (if assigned) as defined in AFO 481 – Loan status codes – Tab General, under ‘Circulation status’.
· Document control - Change History
|
Version |
Date |
Change description |
Author |
|
1.0 |
unknown |
creation |
|
|
2.0 |
August 2006 |
updates for release 2.4.1 build 17 |
|